Hi. Now we'll talk about truncated SVD. We start with the definition 3.5. For any alpha

larger than zero, we define the truncated SVD, TSVD. You could also write this as TSVD

alpha with the parameter alpha here. By A alpha plus is defined as V sigma alpha

plus times U transposed. Recall that A is U sigma V transposed. And where sigma alpha

plus is a diagonal matrix, sigma one, sigma two, sorry, one over sigma one, one over sigma

two, so it looks like the pseudo inverse that we looked at last time. But then we only go

until sigma r alpha, which is an index which we'll discuss in a minute. And the rest of

them will be zeros again, and again, pallet with zero matrices in order to have compatible

dimensions where r alpha, it looks complicated, but it's easy to visualize. But first the

definition. So it's the minimum of L, which is the number of non-vanishing sigma i's.

So the minimum of that number and the maximum j in one up to the minimum of m and n, such

that sigma j is larger than alpha. So what this does is, is the following. Let's maybe

draw this. This is j and this is sigma j. They are ordered, so sigma one is the largest

one, sigma two will be slightly smaller, and so on. It drops down a bit maybe, so it may

look like that. And then at some point they might hit zero and stay zero afterwards. So

one thing to see is that this is L, which is the last index such that the singular values

are still positive. Now we choose some alpha, for example this alpha, this corresponds to

some threshold like this. And the last index such that sigma j is larger than alpha is

this one. This object here, this quantity, the maximal index such that sigma j is larger

than alpha. And we take the minimum of that and L, which is still this, and this is r

alpha. So r alpha is the index such that all singular values up to this index are above

this line alpha. We can pick a different threshold, so for example if we take alpha two like this,

this will be r alpha two. And if we pick by mistake or by chance alpha three so small

that it is smaller than any existing singular value apart from these here, well we just

take L here. So in this case this L will also be r alpha three. Okay so I hope this visual

explanation makes more sense than this slightly complicated looking definition, which of course

we need, but hopefully this makes it clear what it actually means. And with this we can

define the truncated SVD reconstruction, it also depends on alpha of course, L alpha

f by the following L alpha f is defined as a alpha plus applied to the data. So let's

compare this again with what we looked at before. The application of a plus to f is

the minimum known solution and the application of a inverse was the naive inversion. So they

all have the same basic shape, we take a, we do something like an inversion, if we can

actually do inversion by taking the inverse matrix, this is the naive inversion, if a

is not square or for other reasons not invertible we can take the pseudo inverse. This resolves

some problems but not practically ill-posed inverse problems which are bad because of,

because they are infinite dimensional counterparts from which we derive them were ill-posed,

so this doesn't help them, but this truncated SVD could help because what we do here is

we remove all the singular values and we disregard them completely. So it's kind of like saying

if the singular values become too small we set them to zero, so we change the forward

operator and we ignore them completely and we construct this surrogate inverse which

depends on the truncated SVD. Now does this work at all? So we need some theoretical validation

for that probably next week but let me quickly show you that there is an actual benefit to

that. Let's go back to our example, same thing as always, so this is our image, this is a

blurry image, this is a noisy blurry image and we are not committing inverse crimes,

so we're making it as hard as it would realistically be. So we take a slightly different convolution

kernel in order to generate data which is, so we generate data with a different convolution

kernel than the one we are using in our model assumptions. Okay, so we saw that naive reconstruction

does not work and this is a practically built post inverse problem in the sense that it's

not really imposed, so it's a finite dimensional invertible linear equation, so if we make

the noise small enough the naive reconstruction will work and the minimum non-solution doesn't